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PREFACE 


This  technical  report  documents  the  results  of  research  performed  by  the  Fitts 
Human  Engineering  Division,  Crew  Systems  Directorate  of  the  Armstrong  Laboratory  at 
Wright-Patterson  Air  Force  Base,  Ohio.  This  effort  was  accomplished  under  Work  Unit 
ILIRCH32  and  was  funded,  in  part,  by  Armstrong  Laboratory  In-house  Laboratory 
Independent  Research  (ILIR)  Program  funds.  Lt  Col  Frank  Fisher  was  Principal 
Investigator  at  the  inception  of  this  effort.  Ms.  Gloria  Calhoun  assumed  program 
management  responsibilities,  upon  his  departure. 

During  the  course  of  this  research,  Mr.  Matthew  Middendorf  from  Middendorf 
Scientific  Services,  Inc.  provided  systems  analysis  expertise.  Ms.  Darby  Mahan,  imder 
the  AFOSR  High  School  Apprenticeship  Program,  compiled  the  results  from  the 
debriefing  questionnaires.  Several  individuals  working  for  Logicon  Technical  Services,  Inc. 
(LTSI)  were  members  of  the  research  team:  Mr.  David  Ingle  (human  factors),  Mr.  John 
Schnurer  (electronics),  and  Dr.  Victoria  Tepe  Nasman  (neurophysiology).  Dr.  Nasman,  in 
particular,  was  a  key  contributor  and  provided  valuable  technical  insight  from  the 
project’s  inception.  Much  of  the  documentation  contained  herein  was  dravra  from  her 
work  while  she  was  employed  with  LTSI. 
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INTRODUCTION 


After  nearly  three  decades  of  experimental  and  clinical  research  in  the  use  of 
operant  conditioning  and  biofeedback,  it  is  now  well-established  that  human  beings  can 
learn  to  self-regulate  various  aspects  of  their  own  brain  electrical  activity  (Travis,  Kondo 
and  Knott,  1975;  Plotkin,  Mayer  and  Loewy,  1976;  Pressner  and  Savitsky,  1977; 
Kuhlman,  1978;  Fehmi,  1978;  Elbert,  Rockstroh,  Lutzenberger  and  Birbamner,  1980; 
Roger  and  Galand,  1981;  Finley  and  Johnson,  1983;  Rockstroh,  Birbaumer,  Elbert  and 
Lutzenberger,  1984;  Miltner,  Larbig  and  Braun,  1986).  Controlled  experimental  studies 
also  provide  convincing  evidence  that  learned  and  voluntary  self-control  of 
electroencephalographic  (EEG)  or  evoked  potential  (EP)  activity  can  be  achieved  as  a  real 
and  nontrivial  phenomenon,  rather  than  as  the  result  of  artifactual  or  peripheral  mediation 
(Rosenfeld  and  Hetzler,  1978;  Elbert  et  al.,  1980;  Roger  and  Galand,  1981;  Finley,  1983; 
Finley  and  Johnson,  1983). 

Since  the  mid-1980s,  a  number  of  different  methodologies  have  been  tested  to 
examine  the  possibility  that  brain  electrical  signals  might  be  used  to  control  or 
communicate  with  physical  devices.  Preliminary  efforts  have  been  quite  successful  in 
demonstrating  control  and  communication  by  self-regulation  of  alpha/beta  frequency 
activity  (Elder,  Lashley,  Kedouri,  Regenbogen,  Martyn,  Roundtree,  and  Grenier,  1986) 
and  sensori-motor  (SMR  or  mu)  rhythms  (Wolpaw,  McFarland,  Neat,  and  Forneris, 
1991).  Self-regulation  of  the  steady-state  visual  evoked  response  (SSVER)  has  also  been 
used  in  the  Alternative  Control  Technology  Laboratory  of  the  Armstrong  Laboratory  to 
achieve  EEG-based  control  of  the  roll  axis  of  a  flight  simulator  (Junker,  Schnurer,  Ingle, 
and  Downey,  1989;  McMillan,  Calhoun,  Middendorf,  Schnurer,  Ingle,  and  Nasman, 

1995),  the  operation  of  a  functional  electrical  stimulator  (rehabilitation  exercise  device; 
Calhoun,  McMillan,  Morton,  Middendorf,  Schnurer,  Ingle,  Glaser,  and  Figoni,  1995),  a 
computer-based  task  to  change  the  fill  color  of  a  displayed  square  to  match  the  color  of  its 
border  (Nasman,  Ingle,  and  Calhoun,  1995),  and  the  selection  of  switch  icons  (Calhoun 
and  McMillan,  1996). 

As  subjects  achieve  expertise  in  performing  these  tasks,  they  find  it  increasingly 
difficult  to  describe  or  explain  their  own  mental  strategies  for  SSVER  self-regulation. 
Increasing  expertise  seems  to  involve  greater  automaticity,  such  as  subjects  who  report 
only  that  they  “think  left”  or  “think  right”  or  those  who  report  that  their  SSVER  changes 
“almost  automatically”  in  response  to  task  requirements.  This  is  not  unlike  how  one 
might  describe  one’s  mental  strategy  for  achieving  a  simple  arm  or  head  movement.  With 
the  exception  of  such  anecdotal  comments,  demonstrations  of  neural  self-regulation  have 
generally  failed  to  address  the  question  of  what  neuro-cognitive  mechanisms  underlie  the 
reported  phenomena. 
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With  the  increased  emphasis  on  applying  neural  self-regulation  capability  as  a 
“direct  brain  interface”,  there  is  an  increased  need  to  understand  how  neural  self-regulation 
is  accomplished.  It  is  necessary  but  not  sufficient  to  conclude  that  the  control  of  brain 
electrical  activity  can  be  accomplished  by  nontrivial  or  direct  mediation.  Applied  designs 
must  take  into  account  those  neuro-cognitive  phenomena  that  support  or  hinder 
successful  neural  self-regulation.  It  is  therefore  necessary  to  identify  and  describe  its 
mechanisms  as  precisely  as  possible.  In  addition,  it  is  necessary  to  identify  by  what 
strategies  operators  learn  and  manage  neural  self-regulation,  the  importance  of  control 
signal  feedback,  in  what  experimental  and  applied  tasks  these  skills  may  be  most 
successfully  and  efficiently  applied,  and  how  to  maximize  signal  quality  by  effective 
electrode  placement,  application,  signal  recording  and  acquisition. 

OBJECTIVE 

This  research  was  directed  towards  examining  the  nature  of  brain  electrical  events 
that  take  place  during  the  self-regulatory  effort.  Although  our  observations  were  shaped 
both  by  the  specific  character  of  our  recording  and  control  methodology,  it  is  believed  that 
the  results  will  provide  researchers  a  better  understanding  of  various  neuro-cognitive 
events  and  strategies  that  support  or  hinder  neural  self-regulation. 

The  specific  objective  of  this  research  was  to  identify  task  and  system  event- 
related  changes  in  brain  electrical  activity  during  SSVER  self-regulation.  We  have 
approached  this  goal  in  two  ways.  First,  two  pilot  studies  were  conducted  to  examine  in 
what  way,  and  at  what  sites,  the  SSVER  is  altered  at  the  level  of  the  scalp  such  that  its 
resulting  recorded  measure  is  either  “enhanced”  or  “suppressed”  during  self-regulation 
tasks.  Second,  a  more  comprehensive  experiment  was  conducted  to  observe  any  changes 
in  recorded  topographic  EEG  data  that  might  suggest  primary  or  secondary  effects  of 
neural  self-regulation  on  sensory  or  cognitive/attentional  information  processing.  Taken 
together,  these  efforts  represent  an  important  first  step  toward  the  validation  of  a  selected 
methodology,  toward  the  sound  application  of  its  related  tasks  and  systems,  and  toward 
the  development  of  successful  protocols  for  training  and  use  (Nasman,  1996). 

The  following  will  provide  a  brief  description  of  the  system  design  employed  in 
this  research,  along  with  summaries  of  the  two  pilot  studies.  Next,  the  topographic 
evaluation  will  be  reported.  The  majority  of  the  report  is  geared  towards  documenting  the 
task  designed  for  the  latter  study. 
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SSVER  SELF-REGULATION:  SYSTEM  DESIGN 


Studies  at  the  Armstrong  Laboratory  have  focused  on  the  human’s  ability  to 
voluntarily  control  EEG  responses  to  sinusoidally  modulated  light  (McMillan,  et  al., 
1995).  The  experimental  apparatus  simultaneously  controls  the  frequency  of  the  evoking 
stimulus  and  provides  the  user  with  feedback  on  the  strength  of  the  EEG  response  at  that 
frequency.  The  current  system  (Figure  1)  uses  fluorescent  lights  partially  modulated  at 
13.25  Hz  (26%  depth  of  modulation)  to  evoke  a  steady-state  response  in  the  visual 
cortex.  The  signal  used  for  control  is  the  amplitude  at  13.25  Hz  of  the  bipolar 
(differential)  SSVER  recorded  with  gold  cup  electrodes  between  occipital  sites  01  and  02 
(left  and  right  visual  cortical  hemispheres,  respectively,  according  to  the  1 0-20 
International  System  for  surface  electrode  placement;  Jasper,  1958).  The  EEG  signal 
from  the  scalp  electrodes  is  amplified  and  then  synchronously  processed  by  a  lock-in 
amplifier  system.  The  lock-in  amplifier  provides  a  continuous  measure  of  the  magnitude 
of  the  13.25  Hz  SSVER  component  of  the  EEG  signal.  The  magnitude  of  the  response  is 
provided  to  the  subject  feedback  display  and  the  task  control  algorithm. 

Use  of  the  SSVER  amplitude  as  the  control  signal  has  been  applied  to  several 
types  of  tasks  in  our  laboratory  (Calhoun  and  McMillan,  1996).  In  each  of  the  task 
paradigms  investigated,  two  types  of  feedback  are  provided.  First,  changes  in  the  task 
state,  as  a  result  of  the  EEG-based  control,  can  be  observed.  Second,  near  real-time 
feedback  on  the  amplitude  of  the  SSVER  is  provided  in  a  separate  display  element. 

To  date,  only  one-dimensional  control  has  been  attempted.  This  involves 
translating  the  SSVER  amplitude  into  a  binary  control  signal.  The  control  logic  emplo>  s 
threshold  and  duration  requirements  to  permit  stable  control  of  an  external  device  despite 
variability  in  the  operator's  SSVER  control.  One  implementation  produces  a  control 
output  when  the  SSVER  remains  above  or  below  experimenter-specified  thresholds  for 
75%  of  the  samples  in  a  one-half  second  interval.  These  settings  require  the  operator  to 
produce  sustained  changes  in  the  SSVER;  however,  brief  SSVER  fluctuations  do  not 
interrupt  task  control.  The  threshold,  duration  and  percentage  parameters  are  adjustable 
for  individual  subjects  and  for  specific  device  applications. 
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Controlled 


Figure  1.  Illustration  of  EEG-based  control  using  steady-state  visual  evoked  response  self-regulation. 


PILOT  STUDY  1 

SSVER  SELF-REGULATION  AS  A  FUNCTION  OF  ELECTRODE  SITE 

Purpose 

The  purpose  of  this  pilot  study  was  to  consider  the  question  of  whether  the 
SSVER  might  be  recorded  from  multiple  scalp  sites,  rather  than  strictly  from  its 
traditional  source  as  a  signal  derived  from  the  occipital  scalp  (Ingle  and  Nasman,  1994). 
There  is  little  reason  to  assume  that  SSVER  self-regulation  should  be  limited  in  its  basis, 
or  in  its  effect,  to  the  sensory  cortex.  For  example,  learned  volitional  control  over  sensory 
processing  might  invoke  operations  related  to  perceptual-associative  processing  and/or 
high  level  attentional  switching  and  control,  and  thus  provoke  related  changes  in  brain 
electrical  activity  at  other  recording  sites.  This  pilot  study  considered  whether  SSVER 
self-regulation  might  produce  related  changes  that  enable  task  control  from  anterior 
midline  recording  sites.  Of  particular  interest  were  central  and  frontal  midline  scalp 
recording  sites. 


Method 

Across  two  subject  groups,  a  total  of  nine  subjects  were  tested  for  their  ability  to 
enhance  and  suppress  the  SSVER  beyond  criterion  thresholds  in  response  to  task 
commands  (McMillan,  et  al.,  1995).  Each  subject  had  received  some  previous  training 
using  the  occipital  SSVER  to  control  the  experimental  task.  A  task  display  provided 
SSVER  amplitude  feedback  and  instructions  to  enhance  or  suppress  the  SSVER  to  the 
evoking  stimulus.  The  SSVER  was  monitored  on  each  trial  from  bipolar  (differential) 
occipital  (01/02),  central  (C3/C4)  or  frontal  (F3/F4)  electrode  pairs  (left  mastoid  as 
ground).  The  three  recording  configurations  were  independently  calibrated.  In  Subject 
Group  #1,  each  subject  was  tested  with  a  randomly  ordered  series  of  six  68-second  trials 
at  each  recording  site.  Similar  procedures  were  used  for  Subject  Group  #2,  except  only 
four  52-second  trials  were  presented  at  each  recording  site.  All  subjects  were  naive  as  to 
control  site  on  each  trial.  Performance  was  scored  as  correct  responses  to  task 
commands. 


Results 

On  the  average,  control  performance  scores  were  higher  for  trials  in  which  the 
SSVER  was  recorded  at  the  occipital  scalp.  However,  differences  in  performance  among 
the  three  test  recording  sites  failed  to  reach  statistical  significance  in  either  of  the  two 
subject  groups  (p  >  0.3).  Subjects  were  able  to  achieve  some  level  of  control  at  all  sites 
(Figure  2).  On  an  individual  basis,  differences  associated  with  recording  site  were 
apparently  related  to  subjects’  overall  levels  of  ability.  Statistically  significant  “best”  or 
“worst”  recording  sites  (frontal,  central,  or  occipital)  emerged  only  for  subjects  who  failed 
to  achieve  better  than  50%  average  correct  responses  for  some  sites  (Figure  3). 
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Conclusions/Discussion 

These  preliminary  results  suggest  that  SSVER  self-regulation  may  involve  neural 
processes  not  limited  to  those  traditionally  accessed  over  the  occipital  scalp,  and  that 
trained  individuals  may  also  achieve  volitional  control  over  related  activities  at  the  anterior 
midline.  The  data  show  that  individual  subjects  can  generally  achieve  varying  degrees  of 
control  at  frontal,  central,  and  occipital  recordings  sites,  that  is,  effective  and  comparable 
control  can  be  achieved  at  locations  other  than  occipital.  It  still  needs  to  be  determined 
what  phenomena  are  responsible  for  control,  for  example,  at  frontal  versus  occipital  sites. 
Perhaps  the  changes  responsible  for  frontal  control  are  merely  more  distant  (volume- 
conducted)  presentations  of  the  changes  that  support  occipital  control.  Another 
possibility  is  that  the  changes  responsible  for  frontal  control  are  in  some  way  prerequisite 
(e.g.,  attention-related)  to  changes  in  the  occipital  sites.  The  results  may  also  be 
influenced  by  which  electrode  site  was  used  in  the  initial  training  or  the  aecuracy  of  the 
calibration  procedure.  If  further  research  demonstrates  that  control  can  be  reliably 
achieved  at  multiple  sites,  it  is  possible  that  this  capacity  can  be  harnessed  to  implement 
multi-dimensional  control. 


PILOT  STUDY  2 
SSVER  SELF-REGULATION: 

EXAMINATION  OF  AMPLITUDE  VERSUS  PHASE 

Purpose 

The  majority  of  biofeedback  research  has  focused  on  the  control  of  biological 
signal  amplitudes,  while  little  work  has  been  done  on  the  control  of  biological  signal 
timing.  However,  there  are  basic  studies  which  indicate  that  both  the  amplitude  ^ 
phase  of  the  brain  steady-state  evoked  response  are  sensitive  to  attention  and  response 
preparation  (Rockstroh,  Mueller,  Elbert,  and  Makeig,  1994)  and  cognition  (Wilson  and 
O’Dormell,  1986).  SSVER  amplitude  reduction  and  phase  lag  have  also  been  observed  in 
association  with  increased  regional  (e.g.,  frontal)  eortical  activity  related  specifically  to 
heightened  visual  attention  (Silberstein,  Ciorciari,  and  Pipingas,  1995). 

The  SSVER-based  control  signal  in  the  present  system  is  derived  as  a  bipolar 
(differential)  measure  of  occipital  activity  between  hemispheres.  Because  the  EEG  data  is 
acquired  differentially,  subjects  can  change  the  amplitude  of  the  SSVER  by  self-regulating: 
(1)  the  relative  amplitude  of  SSVER  activity  at  the  two  recording  sites,  (2)  the  relative 
timing  (phase)  of  SSVER  activity  at  the  two  recording  sites,  or,  (3)  a  combination  of  both 
signal  characteristics  (see  Figure  4).  The  purpose  of  this  pilot  study  was  to  examine  in 
what  ways  SSVER  activity  is  altered  at  the  level  of  the  scalp  such  that  the  resulting 
control  signal  is  either  enhanced  or  suppressed  (Nasman,  Ingle,  and  Schnurer,  1994). 
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SUPPRESSED 
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Figure  4.  Illustration  of  the  two  possible  mechanisms  for  self-regulating  the 

steady-state  visual  evoked  response,  recorded  with  bipolar  techniques. 

Method 

Four  trained  subjects  received  on-line  bipolar  SSVER  amplitude  feedback  and  a 
randomly  ordered  series  of  task  commands  to  enhance  or  suppress  their  SSVER  to  a 
modulated  light  stimulus.  Matched  amplifiers  were  used  to  acquire  monopolar  (single 
site)  and  bipolar  (differential)  signals  over  the  left  and  right  occipital  scalp  (01/02,  left 


8 


mastoid  as  ground).  Each  subject  performed  SSVER  self-regulation  in  twelve  repeated  50- 
second  task  trials.  Both  monopolar  and  bipolar  signals  were  simultaneously  recorded 
across  the  occipital  midline  to  determine  whether  the  control  signal  was  regulated  as  a 
function  of  hemispheric  differences  in  SSVER  phase  and/or  amplitude. 

The  bipolar  SSVER  signals  for  each  subject  were  reviewed  to  select  several  two- 
second  epochs  that  represented  sustained,  task-related  periods  of  SSVER  enhancement 
and  suppression.  Monopolar  data  from  the  left  and  right  occipital  sites  for  the 
corresponding  epochs  were  ensemble  averaged  and  compared  with  respect  to  phase  and 
amplitude. 


Results 

Each  of  the  four  subjects  tested  showed  evidence  of  phase-based  control.  A 
variety  of  control  mechanisms,  though,  were  evident  in  individual  records  (see  Figure  5). 
SSVER  suppression  was  associated  with  phase  synchrony  in  the  two  monopolar 
channels,  with  occasional  small  and  unsustained  amplitude  differences.  SSVER 
enhancement  was  associated  generally  with  a  sustained  or  periodic  unilateral  phase  shift 
ranging  from  45-180  degrees  in  a  single  channel,  and  was  occasionally  also  associated  with 
sustained  inter-hemispheric  mean  amplitude  differences  (20-50%  signal  range).  The  two 
subjects  who  showed  the  best  and  most  reliable  task  performance  also  showed  the  most 
consistent  and  robust  phase  effects,  whereas  these  same  effects  were  evident  but  less 
distinct  and  less  reliable  for  the  two  subjects  who  did  not  perform  as  well. 

Conclusions/Discussion 

By  examining  the  01  and  02  signals  independently,  several  methods  for  self¬ 
regulating  the  bipolar  SSVER  were  evident.  However,  our  two  most  proficient  subjects 
tended  to  modulate  the  differential  SSVER  amplitude  primarily  by  varying  the  relative 
transport  delay  or  processing  of  sensory  information  at  the  two  sites.  (Data  from  one  of 
these  subjects  is  shown  in  Figure  5b.)  This  may  possibly  reflect  the  synchrony  or 
asynchrony  of  firing  the  underlying  neural  generators  of  the  EEG  at  specific  scalp 
locations.  To  further  examine  the  relationship  between  inter-hemispheric  differences  in 
phase  and  amplitude  during  SSVER  self-regulation,  additional  data  is  required. 

These  results  also  suggest  that  an  alternative  control  algorithm  be  examined.  If 
regulation  of  response  amplitude  plays  a  secondary  or  modulatory  role  in  SSVER  self¬ 
regulation,  then  use  of  a  phase-based  control  algorithm  may  be  more  optimal.  Subjects 
may  achieve  more  reliable  and  precise  control  if  the  algorithm  is  designed  to  employ  phase 
information  only.  However,  it  is  also  possible  that  the  availability  of  both  components  is 
important  to  some  subjects,  or  at  different  points  in  the  learning  process.  Additionally,  it 
would  be  interesting  to  examine  if  SSVER  self-regulation  can  even  be  accomplished  with 
monopolar  recording  of  the  control  signal. 
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(a)  SELF- REGULATING  RELATIVE  AMPLITUDE:  Subject  1 


SSVER 

180*1 

Phase 

135- 

Difference 

90- 

(O1-O2) 

45- 

•'degrees" 

0-1= 

2.0-1 

SSVER 

Amplitude 

1.5- 

Difference 

1.0- ' 

(0i~02) 

0.5 

•Volts" 

0-L 

^  N  . 


^  N 


NO  CONSISTENT 
PHASE 
DIFFERENCE 


CONSISTENT 

AMPLITUDE 

DIFFERENCE 


(b)  SELF -- REGULATING  RELATIVE  TIMING  (PHASE):  Subject  2 
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(c)  SELF- REGULATING  RELATIVE  AMPLITUDE  &  PHASE:  Subject  3 
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Figure  5.  Examples  of  three  control  strategies  for  self-regulating  the  steady-state 
visual  evoked  response.  (Subject  2  actually  shows  a  reversed  control 
amplitude  difference  between  enhanced  and  suppressed  segments,  but  the 
phase  difference  is  so  large,  it  makes  counter  amplitude  effects  negligible.) 
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MAIN  STUDY 

SSVER  SELF-REGULATION:  TOPOGRAPHIC  EFFECTS 


Purpose 

The  second  pilot  study  reported  herein  shows  that  bipolar  SSVER  amplitude 
control  can  be  mediated  as  a  secondary  effect  of  inter-hemispheric  phase  shifts. 

However,  this  finding  is  limited  in  the  sense  that  it  describes  primary  sensory  cortical 
events  that  must  be  understood  as  effects  of  other,  more  fundamental  processes  (e.g., 
attentional  regulation). 

It  may  be  argued  that  SSVER  self-regulation  can  be  viewed  as  a  product  of  two 
stages  of  information  processing:  (1)  feedback  information  processing,  and  (2)  responsive 
alteration  of  neural  state.  In  this  study,  EEG  activity  was  recorded  with  respect  to  each 
stage  of  information  processing  by  examining  temporal  records  of  EEG  activity 
antecedent  (2,  above)  or  consequent  (1,  above)  to  critical  task  events  (delivery  of  SSVER 
feedback  and  control  signal  changes).  The  task  paradigm  and  feedback  delivery  employed 
were  designed  to  provide  sustained  two-second  epochs  during  which  subjects  self- 
regulated  the  SSVER  beyond  criterion  amplitude  thresholds.  Here  we  address  the  need  to 
identify  and  describe  neuro-cognitive  processes  on  the  basis  of  observed  changes  in  the 
topography  of  EEG  components  or  dominant  frequency  bands  during  these  epochs  of 
SSVER  self-regulation. 


Method 

Subjects.  Three  adult  volunteers  (one  male  and  two  females),  ages  19-42  years, 
participated  as  subjects.  Each  subject  had  some  level  of  previous  experience  performing 
SSVER  control  using  other  tasks  tested  in  this  laboratory.  However,  for  the  present 
experiment,  every  subject  received  instructions  (Appendix  A)  and  underwent  a  specific 
training  protocol  to  achieve  criterion  performance  on  the  experimental  task.  Subjects 
included  in  this  study  did  not  have  any  history  of  seizure  disorder  or  visual  loss. 

Recording  and  apparatus.  Subjects  were  seated  facing  a  large  white  projection  screen 
(see  Figure  6).  The  experimental  task  display  (17  x  23  inches)  was  presented  on  the 
screen  at  a  viewing  distance  of  64  inches.  Also  reflected  off  the  screen  was  a  white  light 
delivered  from  two  fluorescent  light  tubes,  sinusoidally  modulated  at  a  rate  of  13.5  Hz 
(26%  depth  of  modulation).  The  evoking  stimulus  was  presented  as  a  normally 
distributed  light  (peak  value  =  0.69  fL)  across  the  height  and  width  of  the  entire  viewing 
screen,  including  the  portion  of  the  visual  field  in  which  the  task  feedback  display  was 
presented.  The  evoking  stimulus  was  thus  impossible  to  avoid  while  performing  the  task, 
but  was  sufficiently  diffuse  so  as  not  to  interfere  with  viewing  the  task  display. 
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Figure  6.  Schematic  of  experimental  apparatus  and  setup. 

The  raw  EEG  control  signal  was  acquired  differentially  using  a  standard  bipolar 
montage,  with  electrodes  placed  at  the  left  and  right  occipital  scalp  (01  and  02;  left 
mastoid  as  ground).  The  EEG  was  filtered  (high  pass  at  0.3  Hz  and  low  pass  at  40  Hz) 
and  amplified.  The  resulting  EEG  was  then  processed  by  a  lock-in  amplifier  system  to 
estimate  the  magnitude  of  the  SSVER  at  the  evoking  frequency  (13.5  Hz). 

During  the  experimental  session,  scalpwide  EEG  (0.1-30  Hz)  was  recorded  using 
an  electrode  cap  (Electro-Cap,  Inc.)  with  electrodes  positioned  at  19  sites  across  the 
scalp,  according  to  10/20  System  placement.  (SSVER  recording/control  electrodes  were 
first  positioned  so  as  not  to  conflict  with  cap  electrodes  positioned  over  the  occipital 
scalp.)  EEG  data  acquisition  for  the  electrode  cap  was  managed  by  a  Bio-Logic  Brain 
Atlas  (Bio-Logic  Systems,  Corp.),  with  signals  amplified  and  3  dB  filter  cutoffs  set  to 
pass  signals  between  0.1  and  30  Hz.  Amplified  signals  were  digitized  at  a  rate  of  200  Hz. 
Signals  were  referenced  to  linked  earlobes.  A  sync  (TTL)  pulse  marker  train  presented  on 
the  Oz  channel  was  used  to  encode  the  EEG  record  to  designate  different  task  events: 


12 


(1)  trial  onset,  (2)  trial  command  type  (“enhance”  or  “suppress”  SSVER),  (3)  SSVER 
modification  made  (enhance  or  suppress  step),  (4)  task  progression,  and  (5)  task 
completion  versus  time  out.  All  electrode  impedances  were  held  below  5  Kohms. 

Experimental  task.  Simulation  timing,  task  control,  and  data  recording  were 
managed  by  a  MicroVax  II.  Image  generation  was  accomplished  using  an  IRIS  3130.  The 
experimental  task  and  feedback  display  were  designed  to  minimize  the  need  for  eye  and 
head  movements  that  can  corrupt  signals  recorded  by  the  brain  mapping  system.  The 
task  also  provided  definitive  task  segments  to  be  used  in  the  analysis  of  data.  The 
computer-based  task  involved  matching  the  fill  color  of  a  square  with  that  of  its  border. 
Subjects  changed  the  shade  of  the  square’s  fill  color  by  self-regulating  and  sustaining  their 
SSVER  above  or  below  calibrated  thresholds.  The  palette  consisted  of  nine  shades 
ranging  from  100%  blue  to  100%  red  (Figure  7).  The  two  outermost  shades  were  reserved 
for  “overshoot”  conditions,  to  be  described  later.  The  three  innermost  shades  served  as 
initial  fill  colors  for  the  square  and  the  remaining  shades  served  as  “command”  colors. 

The  command  colors  were  presented  in  the  border  area  surrounding  the  square  to  inform 
the  subject  on  whether  the  task  required  the  SSVER  to  be  enhanced  or  suppressed.  The 
shades  used  in  the  color  palette  had  a  high  degree  of  contrast  to  increase  the  likelihood 
that  subjects  would  be  able  to  tell  when  the  fill  and  border  colors  matched,  without 
closely  examining  the  square’s  perimeter. 


Initial  Colors 

BLUE  1  1 

2  3 

4 

^  8  i:?y 

Command 

shades  for 
SSVER 
suppress  tasks 

Command 
shades  for 
SSVER 
enhance  tasks 

Figure  7.  Illustration  of  color  palette  consisting  of  nine  shades  ranging  from  100%  blue 
(shade  1)  to  100%  red  (shade  9).  Shades  4,  5,  and  6  were  initial  fill  colors  and 
shades  1  and  9  were  reserved  for  overshoot  conditions. 

The  threshold  values  used  in  the  control  logic  were  calibrated  individually  for  each 
subject  prior  to  testing  and  were  represented  in  the  displayed  square  as  two  fixed  white 
circles.  A  black  circle  provided  feedback  to  the  subjects  on  the  current  amplitude  of  the 
SSVER  in  relation  to  the  two  thresholds.  The  diameter  of  the  black  circle  changed,  in  real¬ 
time,  in  proportion  to  SSVER  amplitude.  Between  thresholds,  the  diameter  of  the  black 
circle  changed  linearly  with  respect  to  changes  in  the  signal  processed  by  the  lock-in¬ 
amplifier;  beyond  thresholds,  the  circle  diameter  changed  nonlinearly  to  prevent  clipping 
of  dynamic  SSVER. 
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The  black  and  white  circles,  respectively  representing  SSVER  feedback  and 
thresholds,  are  illustrated  in  Figure  8,  along  with  how  the  task  format  changed 
corresponding  to  changes  in  the  subject’s  SSVER  amplitude.  In  Figure  8a,  the  subject  s 
task  was  to  enhance  (increase)  the  SSVER  amplitude  to  change  the  square  fill  color  from 
blue  to  red.  When  the  subject’s  SSVER  amplitude  had  exceeded  the  “enhancement” 
threshold,  the  black  circle  was  larger  than  the  outermost  white  circle.  Once  the  criterion 
time  duration  was  met,  the  square’s  fill  color  stepped  one  shade  closer  towards  the  red 
end  of  the  color  palette.  Keeping  the  SSVER  amplitude  above  threshold  for  another 
criterion  time  duration  resulted  in  another  color  shade  step,  such  that  now  the  square’s  fill 
color  matches  the  border.  To  keep  the  fill  color  from  progressing  to  an  even  deeper  red 
shade  (i.e.,  an  overshoot  condition),  the  subject  had  to  suppress  the  SSVER  amplitude 
below  the  enhancement  threshold.  If  the  black  circle  was  between  the  two  white  circles, 
there  was  no  change  in  the  square’s  fill  color. 


SSVER 

Amplitude 

(uV) 


Time  (sec) 


Figure  8a.  Example  time  history  of  SSVER  amplitude  and  corresponding  task  display 
sequence  for  trial  requiring  SSVER  enhancement. 
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In  Figure  8b,  the  subject’s  task  was  to  suppress  (decrease)  the  SSVER  amplitude 
to  change  the  square  fill  color  fi-om  red  to  blue.  When  the  SSVER  amplitude  had 
decreased  below  the  “suppression”  threshold,  the  black  circle  was  smaller  than  the 
irmermost  white  circle.  Upon  meeting  the  time  duration  criterion,  the  square’s  inner  shade 
stepped  towards  blue.  To  avoid  overshooting  the  commanded  color,  the  subject’s  SSVER 
amplitude  had  to  return  above  the  suppression  threshold  after  the  fill  and  border  colors 
matched. 


Figure  8b.  Example  time  history  of  SSVER  amplitude  and  corresponding  task  display 
sequence  for  trial  requiring  SSVER  suppression. 

Task  procedures.  In  each  trial,  the  subject  began  by  viewing  a  simple  white  outline  of 
the  task  display;  this  was  presented  for  a  duration  of  one  second  to  allow  the  subject 
time  to  focus  on  the  proper  area  of  the  projection  screen  prior  to  task  onset.  Next,  the 
square  was  filled  with  one  of  the  three  (randomly  selected)  initialization  colors  from  the 
middle  range  of  the  color  palette.  The  circle  representing  SSVER  amplitude  was  also 
displayed  and  updated  in  real-time  for  one  second.  This  provided  time  for  the  subject  to 
monitor  the  SSVER  amplitude  and  view  the  initial  color  of  the  square,  before 
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the  area  around  the  square’s  border  was  filled  with  the  task  command  color.  After  this 
time  period,  and  after  the  SSVER  amplitude  fell  within  the  “neutral  zone,”  the  command 
color  was  presented  in  the  border  (set  to  a  color  shade  away  from  the  square’s  initial  fill 
color;  one  or  two  steps  on  the  color  palette  in  training  and  two  steps  in  the  final 
experimental  session).  The  subject’s  task  was  then  to  self-regulate  his  or  her  SSVER 
amplitude  in  the  direction  necessary  to  match  the  fill  color  of  the  square  to  the  color  of 
the  square’s  border.  Each  change  in  color  shade  required  maintaining  the  SSVER 
enhancement  or  suppression  with  respect  to  the  experimenter-specified  thresholds  for  the 
criterion  time  duration.  If  the  SSVER  amplitude  temporarily  crossed  into  the  neutral  zone 
between  thresholds,  the  time  clock  was  not  reset.  However,  if  the  SSVER  amplitude 
crossed  over  both  thresholds,  then  the  duration  clock  was  reset  to  zero  and  a  new 
criterion  time  period  was  begun. 

Once  a  color  match  was  achieved,  the  subject  was  required  to  stop  the  color¬ 
sequencing  by  reversing  the  SSVER  amplitude  towards  the  opposite  threshold.  If  the 
subject  failed  to  reverse  the  SSVER,  the  square’s  fill  color  continued  to  change  to  a  deeper 
shade  in  the  designated  direction  and  “overshoot”  the  command  color.  In  this  case,  the 
subject  had  to  regain  control  over  the  SSVER  signal  and  reacquire  the  color  match  by 
modifying  SSVER  amplitude  in  the  opposite  direction.  The  purpose  for  enabling 
dynamic  control  was  to  help  prevent  subjects  from  “gaming”  the  task.  Rather  than  have 
subjects  simply  maximize  or  minimize  their  SSVER  when  they  realized  that  the  task 
required  an  enhancement  or  suppression,  we  wanted  subjects  to  acquire  the  ability  to 
exercise  more  precise  and  rapid  control  of  the  SSVER.  With  the  possibility  of 
overshooting  the  color,  subjects  had  to  maintain  dynamic  control  of  the  SSVER 
amplitude. 

Upon  successful  task  completion,  a  reinforcing  feedback  display  was  presented 
consisting  of  multi-colored  flashes.  If  the  task  was  not  successfully  completed  within  the 
specified  time  limit,  text  was  displayed  which  indicated  that  the  trial  timed  out.  After 
each  series  of  twelve  trials,  a  brief  scoring  display  was  presented  which  provided 
information  on  the  number  of  successful  trials  (color  matches),  etc. 

Trials  were  separated  by  a  one-second  pause.  Trials  were  delivered  in  a  block  of 
twelve,  each  block  separated  by  a  two  minute  break.  Between  two  and  three  blocks  were 
presented  in  a  daily  session.  In  each  block  of  twelve  trials,  six  trials  required  SSVER 
enhancement  and  six  trials  required  SSVER  suppression. 

Training  sessions.  Before  participating  in  the  final  experimental  session,  involving 
scalpwide  data  collection,  subjects  had  to  achieve  the  following  in  training: 

a)  perform  trial  blocks  in  which  all  trials  required  two  changes  in  fill  color  shade, 


b)  perform  the  trial  within  30  seconds, 

c)  perform  the  trial  with  demanding  control  parameters  (sustained  SSVER  with 
1 00%  of  the  samples  past  threshold  for  two  seconds  for  each 

color  shade  step),  and 

d)  at  the  above  control  parameters,  successfully  complete  (match  colors)  80%  of 
trials,  averaged  over  the  trial  blocks  in  each  of  three  consecutive  training  sessions. 

The  very  stringent  control  settings  were  required  for  the  final  experimental  session 
to  facilitate  collection  of  scalpwide  broad-band  EEG  during  sustained  periods  of  SSVER 
enhancement  and  suppression.  To  assist  subjects  in  attaining  this  level  of  performance, 
initial  training  sessions  required  that  only  80%  of  the  samples  in  a  one  second  interval 
meet  the  threshold  criterion  to  produce  a  color  step.  Additionally,  subjects  were  given  60 
seconds  to  complete  the  trial  and  some  trials  only  required  one  color  step,  as  opposed  to 
two  color  steps  from  the  initial  color  shade.  Both  “one  step”  and  “two  step”  color 
matches  were  used  in  training  to  force  subjects  to  exercise  precise  control.  This  procedure 
also  helped  prevent  subjects  from  learning  an  automatic  response  that  may  be  inherently 
different  from  the  dynamic  control  that  is  involved  when  the  SSVER  response 
requirements  change  from  trial  to  trial. 

As  subjects  acquired  proficiency  at  these  levels,  the  trial  length  was  decreased  and 
the  control  settings  (duration  and  percentage  requirements)  were  gradually  increased. 
Subjects  completed  a  minimum  of  ten  training  sessions,  with  the  task  requirements 
increasing  in  difficulty,  as  their  performance  improved.  During  the  last  few  training 
sessions,  all  trials  required  two  color  steps  to  achieve  a  match. 

Experimental  session.  In  the  final  session,  a  broad-band  recording  system  was  used  to 
collect  scalpwide  electrical  activity  during  sustained  periods  of  self-regulation  of  the 
SSVER.  Each  trial  during  the  experimental  session  required  two  successive  correct  color 
steps  to  match  the  command  color. 

Besides  the  multi-channel  electrode  cap,  subjects  were  instrumented  for  recording 
the  SSVER  control  signal.  Subjects  were  instructed  to  sit  as  still  as  possible  and  to 
minimize  eye  movement  and  blinking.  Every  subject  was  then  tested  in  one  baseline  and 
two  experimental  conditions.  In  the  baseline  condition,  subjects  viewed  the  static  outline 
of  the  task  display  for  30  seconds.  This  condition  was  included  to  allow  the  experimenter 
to  observe  on-line  EEG  and  to  address  issues  of  subject  cooperation  and  artifact.  After 
obtaining  baseline  data,  four  blocks  (12  task  trials  each)  were  presented,  two  blocks  with 
each  of  two  experimental  conditions: 
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Condition  1 :  Procedures  identical  to  those  employed  in  the  training  sessions  were  used. 
The  feedback  was  mapped  consistently  to  changes  in  the  amplitude  of  the  SSVER  control 
signal.  These  “experimental  trials”  were  mapped  as  follows: 


1  EXPERIMENTAL  TRIALS  I 

TASK 

COMMAND 

SSVER  CHANGE 
REQUIRED 

FEEDBACK 

PROVIDED 

RESULTING  CHANGE  IN 
SQUARE  FILL  COLOR 

red  border 

enhance 

turns  red 

blue  border 

suppress 

circle  shrinks 

turns  blue 

Condition  2:  Trials  were  randomized  such  that  fifty  percent  of  the  trials  delivered 
feedback  consistently  mapped  with  SSVER  amplitude,  as  used  in  Condition  1 .  The  other 
trials  provided  inaccurate  (noncontingent)  real-time  amplitude  feedback  (circle  size).  The 
following  mapping  was  used  with  these  “control”  trials: 


1  CONTROL  TRIALS  | 

TASK 

COMMAND 

SSVER  CHANGE 
REQUIRED 

FEEDBACK 

PROVIDED 

RESULTING  CHANGE  IN 
SQUARE  FILL  COLOR 

red  border 

enhance 

circle  shrinks 

turns  blue 

blue  border 

suppress 

turns  red 

During  this  control  condition,  subjects  were  aware  that  circle  size  would  be  an  inaccurate 
source  of  feedback.  The  purpose  of  including  this  noncontingent  feedback  condition  was 
to  observe  any  changes  that  the  absence  of  accurate  feedback  might  produce  in 
performance  and  EEG. 

Following  completion  of  the  four  blocks  of  trials,  subjects  completed  a  final 
debriefing  questionnaire  that  addressed  their  strategies  or  techniques  for  self-regulating 
their  SSVER  and  their  assessment  of  different  aspects  of  completing  the  color  matching 
task  (see  Appendix  B).  The  final  experimental  session  lasted  approximately  two  hours. 

Data  analyses.  Scalpwide  EEG,  task  performance  data,  and  subjective  questionnaire 
responses  were  collected  to  assess  EEG  topography,  subject  performance,  and  opinions 
regarding  the  experimental  task. 

Topographic  Brain  Mapping:  Scalpwide  EEG  from  19  sites  was  processed  off-line 
to  identify  topographic,  frequency,  or  event-related  changes  corresponding  to  SSVER 
self-regulation.  Trials  that  contained  eye  or  body  movements,  or  other  recording 
artifacts,  were  excluded  from  this  analysis.  Fast  Fourier  transforms  were  performed  on 
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remaining  trials,  processed  as  two-second  epochs  preceding  each  color  step 
(corresponding  to  the  2-second  beyond  threshold  requirement).  SSVER  enhancement 
and  suppression  trials  were  analyzed  separately.  Ensemble  averages  were  derived 
separately  for  different  categories  of  task  and  trial  parameters  (evoke  versus  suppress 
commands,  correct  versus  incorrect  responses,  experimental  versus  control  trials).  A 
total  of  79  artifact-free  epochs  were  included  in  the  analysis. 

Task  Performance  Data:  Performance  in  the  two  experimental  conditions  was  compared 
by  examining  three  measures.  First,  the  average  number  of  steps  (color  shade  changes) 
made  to  complete  the  trials  was  calculated.  At  least  two  steps  were  required  for  all  the 
trials  presented  in  the  final  session.  Averages  greater  than  2.0,  then,  reflect  the  subject 
making  erroneous  steps  (e.g.,  modifying  the  SSVER  in  the  direction  opposite  what  was 
commanded,  resulting  in  a  step  towards  the  wrong  end  of  the  color  palette,  and/or 
overshooting  the  commanded  color;  both  error  types  necessitated  additional  steps  to 
acquire  the  color  match).  The  second  measure  was  average  time  to  complete  the  trial 
(subjects  had  a  maximum  of  30  seconds  to  complete  each  trial).  The  third  measure  was  the 
average  percentage  of  color  matches  successfully  acquired  in  each  condition. 

Questionnaire  Data:  Subjective  data  collected  through  the  debriefing  questionnaires 
were  compiled  to  be  presented  in  tabular  form. 

Results. 

Topographic  brain  mapping.  The  EEG  power  spectra  that  resulted  from  these  recording 
sessions  illustrated  a  clear  and  dominant  peak  at  the  13.5  Hz  evoking  frequency.  This 
suggests  that  the  SSVER  control  demonstrated  reflects  subjects’  self-regulation  of  a  narrow 
frequency  band  rather  than  broad-band  artifactual  activities.  Different  topographical 
distributions  were  also  observed  for  “enhance”  versus  “suppress”  trials  (Nasman,  Ingle, 
Calhoun,  1995).  The  differences  were  primarily  noted  over  the  occipital  scalp,  at  the  13.5  Hz 
evoking  frequency  (Figure  9).  Task-related  SSVER  suppression  was  associated  with  an  even 
and  bilateral  distribution  of  13.5  Hz  activity  across  the  occipital  scalp.  By  contrast,  the  13.5 
Hz  frequency  band  was  lateralized  to  the  left  occipital  hemisphere  during  SSVER 
enhancement.  These  patterns  were  absent  or  attenuated  for  incorrect  color  steps. 
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Enhance  SSVER 


O,  O2 


135 

/ 

■  / 

.  ..  y _ ^ 

n=1 1  trials 


Subject  #1 


Suppress  SSVER 


n=12  trials 


Enhance  SSVER 


Figure  9a.  Topographic  maps  of  13.5  Hz  activity  as  recorded  during  task-related 
enhancement  and  suppression  for  Subjects  1  and  2. 
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Enhance  SSVER 


n=13  trials 


Subject  #3 

Suppress  SSVER 


Figure  9b.  Topographic  maps  of  13.5  Hz  activity  as  recorded  during  task-related 
enhancement  and  suppression  for  Subject  3. 


Task  performance.  Average  trial  scores  and  standard  deviations  (based  on  24  trials  per 
condition,  per  subject)  are  shown  in  Figure  10  for  each  of  the  three  task  performance 
measures.  The  percentage  of  color  matches  acquired  were  generally  less  than  that 
recorded  during  the  last  three  training  sessions  (where  an  80%  average  was  prerequisite  to 
the  final  experimental  session).  This  performance  decline  was  probably  due  to 
distractions  caused  by  the  scalpwide  EEG  recording  used  in  the  final  session.  Also, 
subject’s  performance  on  the  color  matching  task  was  not  as  good  as  that  observed  in 
other  task  paradigms  using  EEG-based  control  (Calhoun  and  McMillan,  1996).  This 
probably  reflects  the  more  stringent  control  settings  used  in  the  present  study  to  facilitate 
collection  of  scalpwide  broad-band  EEG  during  sustained  epochs  of  SSVER  enhancement 
and  suppression.  With  different  settings  (e.g.,  reducing  the  duration  requirement  to  one 
second),  task  performance  would  improve.  Although  the  control  algorithm  used  in  the 
present  study  is  not  optimized  for  task  performance,  its  use  demonstrates  that  subjects 
can  sustain  SSVER  modifications  for  more  than  two  seconds. 
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Condition 

Number  of  Steps 

Trial  Time 

%  Matches 

2.87  (+/- 1.14) 

22.0  (+/-8.7) 

54% 

Control 

3.36  (+/-1.21) 

23.4  (+/-9.0) 

50% 

2 

3.00(+/-1.21) 

15.8  (+/-9.4) 

79% 

Control 

4.17  (+/-1.97) 

24.2  (+/-8.6) 

38% 

3 

2.83  (+/-1.15) 

14.0  (+/-7.9) 

88% 

Control 

4.58  (+1-2.51) 

20.2  (+/-9.2) 

58% 

Figvire  10.  Tcisk  performance.  Average  trial  scores  (+/- standard  deviation) 
based  on  24  trials  per  condition,  per  subject. 

Even  though  the  task  performance  data  recorded  in  the  present  study  are  not  an 
accurate  reflection  of  how  subjects  could  perform  this  EEG-based  control  task  with 
optimal  parameter  settings,  these  data  are  useful  in  comparing  the  two  experimental 
conditions.  For  all  three  subjects  and  on  all  recorded  measures,  performance  was  better 
with  the  experimental  condition  in  which  the  feedback  mapping  was  identical  to  that  used 
in  training.  For  the  control  condition  in  which  SSVER  feedback  was  inconsistently 
mapped  to  the  color  step,  task  performance  was  degraded,  reflected  in  a  decrease  in  the 
percentage  of  successful  color  matches  and  an  increase  in  the  number  of  steps  made  and 
trial  completion  time. 

Debriefing  questionnaire.  Subject  responses  to  the  debriefing  questionnaire  are 
provided  in  Appendix  B.  The  responses  indicate  the  subjects  felt  confident  in  their 
ability  to  perform  the  color  matching  task  by  self-regulating  their  SSVER.  On  a  question 
that  asked  them  to  rate  their  present  overall  ability  to  do  the  task  on  a  scale  from  0  (no 
control)  to  10  (perfect  control),  subjects  gave  ratings  of  “7”  (Subject  2)  and  “8”  (Subjects 
1  and  3).  On  questions  addressing  what  strategies  or  techniques  they  used  to  perform  the 
task,  the  subjeets  mentioned  visualization  and  focusing  attention  on  specific  areas  of  the 
task  display.  The  ratings  comparing  the  difficulty  of  learning,  initiating,  sustaining,  and 
stopping  an  enhaneed  SSVER  versus  a  suppressed  SSVER  varied.  Results  show  that  the 
subjects  always  found  one  of  the  two  processes  (enhancing  versus  suppressing  the 
SSVER)  to  be  harder  —  which  one,  however,  differed  on  an  individual  basis. 
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Conclusions/Discussion 

Neural  mediation  established.  Peripheral  mediators  (e.g.,  broad-band  eye  or  muscle 
artifacts)  are  unlikely  with  an  EEG-based  interface  employing  self-regulation  of  a  narrow- 
band  frequency,  the  SSVER  to  an  evoking  stimulus.  Moreover,  the  performance  data 
supports  the  hypothesis  that  SSVER  self-regulation  is  a  nontrivial,  neurally  mediated,  and 
learned  phenomenon.  In  the  control  condition  that  presented  trials  in  which  the  feedback 
was  not  systematically  related  to  the  direction  of  change  in  the  SSVER  amplitude, 
performance  was  degraded  for  all  three  performance  measures.  If  neural  self-regulation  is 
either  a  trivially-mediated  or  a  highly  adaptive  phenomenon,  task  performance  would  not 
have  shown  a  consistent  degradation  in  performance  across  measures  and  for  all  subjects. 
Rather,  the  performance  data  support  the  conclusion  that  the  neural  response  is  learned 
and  that  trained  feedback  association  is  critical. 

The  scalpwide  EEG  data  also  provide  evidence  that  SSVER  self-regulation  is 
neither  trivially  mediated  nor  spuriously  achieved.  Topographic  maps  of  baseline  and 
task-related  records  show  SSVER  activity  over  the  occipital  scalp  as  well  as  over  the  right 
fronto-central  cortices.  Moreover,  the  occipital  SSVER  amplitude  gradients  showed 
differences  reliably  related  to  specific  task  requirements  to  enhance  or  to  suppress  the 
bipolar  SSVER  control  signal. 

Hemispheric-based  neurological  mechanism  suggested.  It  is  well  beyond  the  scope 
of  the  present  effort  to  definitively  establish  the  neurophysiological  basis  for  SSVER  self¬ 
regulation.  The  specific  objective  of  the  present  study  was  to  identify  task  and  system 
event-related  changes  in  brain  electrical  activity  recorded  scalpwide  during  SSVER  self¬ 
regulation.  The  topographic  data  showed  clear  hemispheric  differences  for  occipital 
evoked  response  frequency  distributions  recorded  during  SSVER  enhancement  versus 
suppression.  These  effects  are  consistent  with  the  possibility  that  SSVER  self-regulation 
involves  mediation  of  the  SSVER  between  the  two  bipolar  (differential)  recording  sites. 
Moreover,  the  data  from  the  second  pilot  study  suggest  that  SSVER  amplitude  variation 
during  EEG-based  control  is  probably  the  result  of  some  combination  of  both  signal  phase 
and  amplitude  variation. 

Use  of  bipolar  signal  recording  may  increase  the  flexibility  of  system  design  with 
respect  to  individual  cognitive  strategies.  However,  it  is  also  possible  that  the  bipolar 
recording  may  facilitate  a  strategy  based  on  manipulating  hemispheric  differences,  even 
though  there  is  a  more  optimum  alternative  strategy  based  on  monopolar  or  other 
recording  derivations.  In  that  it  is  impossible  to  know  whether  the  existing  bipolar 
recording  methodology  is  an  asset  or  a  hindrance  to  optimal  learning  and/or  performance, 
alternative  recording  methodologies  for  EEG-based  control  should  be  examined. 
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OUTSTANDING  ISSUES  AND  RESEARCH  QUESTIONS 

Operator  Issues 

Individual  differences  may  be  less  of  a  problem  in  SSVER  recording,  compared  to 
other  brain  electrical  activity,  for  use  in  EEG-based  interfaces.  Because  the  SSVER  is  a 
primary  sensory  response  to  an  exogenous  stimulus,  it  is  recorded  very  reliably  over  the 
primaiy  occipital  cortex.  In  this  laboratory,  it  also  has  been  demonstrated  that  the 
SSVER,  as  well  as  task-related  changes  in  its  amplitude,  may  be  recorded  at  other  (e.g., 
frontal,  central)  sites  along  the  midline  scalp.  This  is  useful  to  know,  in  the  event  that 
future  applications  require  multiple  scalp  recordings  to  support  one  or  more  signal 
controls  or  parameters. 

To  the  extent  that  inter-hemispheric  differences  in  SSVER  timing  and/or  amplitude 
reflect  learned  strategies,  it  may  be  useful  to  observe  their  development  over  training  and 
exposure.  Additionally,  the  ability  to  maintain  SSVER  self-regulation  skills  over  time  has 
not  been  systematically  evaluated.  Perhaps  even  more  important  to  application  issues  is 
whether  operators  can  exercise  SSVER  self-regulation  with  an  EEG-based  interface  in 
complex  task  environments  where  there  are  concurrent  tasks  to  be  performed  and 
associated  distraetions. 


Design  Issues 

It  is  recommended  that  experimentation  continue  to  evaluate  time-locked,  task 
related  effects  of  neural  self-regulation  on  related  cognitive  and  brain  processes. 

Candidate  variables  to  examine  include:  hemispheric  dominance,  phase  versus  amplitude 
modulation,  phase  versus  amplitude  feedback,  attentional  manipulations,  “trivial” 
mechanisms  (e.g.,  gaze,  contrast  or  edge  effects),  and  effects  on  related  and  unrelated  brain 
processes  and  cognitive  operations.  Various  evoking  stimulus  modalities  (visual,  tactile, 
etc.)  should  be  compared.  The  results  of  these  investigations  may  provide  insight  as  to 
how  to  differentiate  between  intentional  and  unintentional  EEG  changes,  improve  the 
precision,  reliability  and  speed  of  control,  and  permit  multi-channel  control. 

The  color  matching  task  and  associated  procedures  used  in  the  study  proved 
acceptable  in  terms  of  task  performance,  subjects’  subjective  opinion,  and  delineation  of 
sustained  epochs  of  SSVER  modifications  for  data  analyses.  However,  performance  on 
the  color  matching  task  during  the  experimental  session,  which  involved  scalpwide  data 
collection,  was  degraded  compared  to  that  recorded  in  the  final  training  sessions.  This 
result,  coupled  with  the  contamination  of  numerous  epochs  by  noise  from  eye 
movements,  significantly  reduced  the  number  of  data  samples  suitable  for  analysis.  For 
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future  experimental  application  of  this  task,  it  is  recommended  that  multiple  sessions  be 
conducted  with  the  scalp  wide  recording  system  in  place,  so  that  subjects  can  become 
more  accustomed  to  its  presence.  Further,  it  is  suggested  that  the  task  display  be  reduced 
in  size  so  as  to  minimize  the  magnitude  of  any  eye  movements.  The  strategy  employed 
by  some  subjects  to  focus  eye  position  on  specific  areas  of  the  task  display  while  self¬ 
regulating  the  SSVER  also  warrants  further  evaluation  to  determine  if  eye  position  or  a 
task  display  variable  is  a  factor  in  subjects’  SSVER  control. 


SUMMARY 

Although  additional  research  is  required  to  fully  understand  the  neuro-cognitive 
basis  for  direct  brain  interfaces,  the  research  presented  in  this  report  sheds  light  on  the 
neural  mechanisms  of  SSVER  self-regulation.  For  example,  the  use  of  amplitude 
modulation,  phase  modulation,  and  their  combination  was  clearly  demonstrated  in  these 
data.  The  findings  reported  herein  also  illustrate  that  brain  electrophysiological  recording 
provides  noninvasive  access  to  neural  events  that  occur  during  SSVER  self-regulation. 
Information  acquired  on  signal  alterations,  as  a  result  of  self-regulation,  may  provide  the 
basis  for  improved  system  control.  Real-time  monitoring  of  various  neuropsychological 
parameters  of  neural  self-regulation  may  enable  an  adaptive  EEG-based  interface, 
increasing  the  accuracy  and  precision  of  system  operation  and  control.  To  the  extent  that 
identified  neural  activities  provide  an  objective  measure  to  track  improvement  in 
performance,  research  efforts  along  these  lines  may  also  provide  a  means  to  greater 
efficiency  in  operator  training. 

Real-world  application  of  EEG-based  interfaces  will  certainly  begin  before  this 
new  technology  is  explored  comprehensively  within  the  laboratory.  Nevertheless,  basic 
and  applied  research,  testing,  and  evaluation  in  the  laboratory  must  continue  to  support 
the  development  of  more  efficient  and  reliable  methodologies  and  systems.  Even  as 
applied  systems  are  conceived  and  implemented,  laboratory  and  applied  observations  will 
provide  essential  research  and  development  support  in  parallel.  Research  concerned  with 
the  nature  of  brain  electrical  events  that  take  place  during  the  exercise  of  EEG-based 
interfaces  is  key  to  optimizing  their  implementation  and  application. 
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APPENDIX  A 


SUBJECT  INSTRUCTIONS:  Color-matching  Task 

In  this  experiment,  you  will  be  tested  for  your  ability  to  control  a  color-matching  task  by 
modifying  your  brain’s  response  to  a  flashing  light.  The  flashing  light  will  be  presented  across  a 
large  projection  screen.  The  task  display  will  be  presented  in  the  center  of  the  projection  screen; 
you  must  attend  to  this  display  in  order  to  do  the  task. 

The  task  display  is  presented  as  a  square.  The  goal  of  the  task  is  to  match  the  fill  color  of  the 
inner  square  with  that  of  its  border.  The  border  color  is  your  TARGET  color.  The  inner 
square  color  will  change  in  response  to  the  strength  of  your  brain’s  response  to  the  flashing  white 
projection  screen. 

•  RED  target  shades  designate  that  you  should  enhance  your  brain’s  response 

•  BLUE  target  shades  designate  that  you  should  suppress  your  brain’s  response. 

There  are  several  shades  of  each  color.  You  should  modify  your  brain’s  response  in  the  direction 
designated  by  the  target  color,  and  you  must  continue  until  the  color  shade  of  the  inner  box 
matches  that  of  the  border. 

Within  the  square  are  three  circles.  There  is  one  black  circle;  the  size  of  this  circle  will  change  in 
proportion  to  the  strength  of  your  brain  response.  As  your  response  increases  in  strength,  the 
black  circle  will  grow  larger;  as  your  response  decreases  in  strength,  the  black  circle  will  become 
smaller.  If  your  goal  is  to  reach  a  red  target  color,  you  will  attempt  to  make  the  black  circle  larger. 
If  your  goal  is  to  reach  a  blue  target  color,  you  will  attempt  to  make  the  black  circle  smaller. 

(blue) 

target  color 
control  response 

—  control  color 


The  white  circles  designate  control  thresholds.  When  the  black  circle  grows  larger  than  the 
outermost  white  circle,  this  means  you  have  achieved  the  response  level  necessary  to  produce  a 
color  change  toward  red.  If  the  black  circle  shrinks  and  becomes  smaller  than  the  innermost 
white  circle,  this  means  you  have  reached  the  low  response  level  needed  to  produce  a  color 
change  toward  blue. 

Note  that  you  must  hold  the  black  circle  beyond  the  appropriate  threshold  for  a  specified 
period  of  time  (0.5  -  2.0  seconds)  in  order  to  produce  a  single  color  step/shade  change.  The 
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experimenter  will  tell  you  about  your  required  “hold”  duration  prior  to  each  training  or  test 
session.  If  you  fail  to  maintain  your  response  level  such  that  the  black  circle  changes  direction 
and  crosses  back  over  both  thresholds  (white  circles),  this  will  re-set  the  computer  clock  to  zero 
and  a  new  “hold”  period  will  begin.  (Crossing  temporarily  into  the  “neutral”  zone  between 
thresholds  does  not  force  the  clock  to  re-set.) 

As  you  maintain  your  desired  response  level,  the  color  of  the  innermost  box  will  step  through 
consecutive  shades  of  color  approaching  the  target  color.  After  you  achieve  the  desired  color 
match,  you  must  return  the  black  circle  to  its  “neutral”  zone  between  the  two  white  circles.  You 
must  accomplish  this  within  the  designated  “hold”  time.  If  you  fail  to  reach  the  neutral  zone  in 
that  time,  the  color  of  the  innermost  box  will  continue  to  change  and  you  will  “overshoot”  the 
intended  color  match.  In  order  to  succeed  after  an  “overshoot,”  you  must  reverse  course  and 
restore  the  innermost  box  to  the  target  color. 

In  each  trial,  you  are  allowed  30-60  seconds  to  achieve  the  color  match  and  return  the  black  circle 
to  its  neutral  position.  (The  experimenter  will  instruct  you  as  to  your  exact  trial  times  during 
training  and  testing.)  If  you  succeed,  the  task  display  will  freeze  and  flash  colorfully  for  a  short 
time  before  displaying  your  trial  score.  If  you  fail  to  acquire  the  target  in  time,  the  trial  will 
simply  end  (“trial  timed  out”).  There  will  follow  a  brief  scoring  display,  which  you  need  not 
consider,  attend,  or  memorize.  This  information  is  displayed  only  to  assist  the  experimenter. 

Trials  will  be  delivered  in  series  of  12  (total  time  =  approximately  7  minutes).  After  each  batch 
of  12  trials,  a  brief  scoring  display  will  summarize  your  performance  in  successful  trials  over  the 
preceding  trial  series.  Most  of  this  information  is  presented  to  assist  the  experimenter.  You  need 
not  memorize  any  of  the  scoring  information,  and  you  may  elect  to  ignore  it  if  you  wish. 
However,  you  are  encouraged  to  note  the  number  of  successful  trials  achieved,  as  this  will  help 
you  to  track  your  improvement  over  trial  series  and  training  sessions. 


SCORE  SUMMARY 

SUCCESSFUL  TRIALS  (%)  =  75%  (9/12) 

AVERAGE  PERCENT  CORRECT  STEPS  =  95.6 

AVERAGE  TRIAL  TIME  =11.2 

AVERAGE#  OF  SPURIOUS  EVOKES  =  0.2 

AVERAGE  #  OF  SPURIOUS  SUPPRESSES  =  0. 1 


Between  each  series  of  12  trials,  you  will  he  given  a  2-minute  break  to  rest  your  eyes. 
Please  alert  the  experimenter  if  you  need  to  take  a  break  at  any  other  time,  or  if  you  are 
experiencing  difficulties  or  discomfort.  Please  tell  the  experimenter  if  you  are  confused 
about  the  task,  or  if  you  need  additional  information  about  how  it  works. 
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APPENDIX  B 


QUESTIONNAIRE:  COLOR-MATCHING  TASK 


The  purpose  of  this  questionnaire  is  to  collect  information  about  your  experience  with 
brain-actuated  control  in  the  color-matching  task.  Your  answers  will  be  used  to  help  us 
evaluate  how  our  experimental  subjects  learn  and  achieve  brain-actuated  control.  Your 
candid  opinions  and  observations  are  a  valuable  contribution  to  this  effort. 

In  some  cases,  we  invite  you  to  answer  questions  in  your  own  words.  Others  are 
answered  by  selecting  a  specific  response  or  checking  a  box  that  best  corresponds  to  your 
experience.  Feel  free  to  contribute  additional  comments,  however,  and  be  as  specific  as 
possible. 

Note:  some  of  the  questions  below  refer  to  "evoke"  and  "suppress"  trials.  "Evoke"  trials 
were  those  in  which  you  received  a  red  command  color  and  had  to  produce  an  increase  in 
your  brain  response  amplitude;  if  you  succeeded,  the  black  circle  grew  larger.  "Suppress" 
trials  were  those  in  which  you  received  a  blue  coinmand  color  and  had  to  decrease  your 
brain's  response  amplitude,  causing  the  black  circle  to  shrink. 

Feel  free  to  request  clarification  if  needed  while  completing  this  questionnaire. 


1.  Describe  your  strategy(ies),  technique(s),  or  approach  to  performing  the  color  matching 
task.  What  did  you  do? 

*  #J,:  I  “visualized”  where  I  wanted  the  real-time  feedback  ring  to  be. 
Thus,  my  attention  was  continually  on  the  feedback  ring  and 
positioning  my  eye  line-of-sight  on  the  desired  location.  I  watched 
the  color  steps/color  command  in  my  periphery.  Also,  I  concentrated 
on  relaxing;  if  I  tensed  up  I  couldn’t  do  anything. 

#2:  Evoking  I  think  about  the  color  red  &  try  not  to  think  of 
anything  else.  Suppressing"^  No  actual  thought,  per  se—  just  a 
mental  sensation  of  “blocking”  activity  in  left  hemisphere.  (I  don’t 
feel  anything-  I  just  describe  it  to  myself  in  this  way,  to  remind 
myself  what  to  do.)  [I]  also  try  to  be  prepared  to  “back  out”  after  a 
match,  so  as  not  to  overshoot. 

#3:  Evoke:  looked  on  the  right  of  the  target 
Suppress:  looked  on  the  left  of  the  target 

*  [Numbers  1-3  before  comments  and  in  tables  refer  to  Subject  1,  2,  and  3.] 
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2.  Did/does  your  technique  differ  depending  on  trial  type  (color  command)?  ...  did  you  do 
something  different  to  "evoke"  versus  "suppress"  your  brain's  response?  Please  be  as 
specific  as  possible. 

#1:  For  suppress  trials,  I  concentrated  on  making  the  feedback  ring  a 
dot,  which  involved  staring  at  the  center  of  the  square,  and  thinking 
“small.”  For  evoke  trials,  I  visualized  a  large  black  outer  ring  (and 
thus  my  eyes  were  directed  outside  the  outer  ring).  Also,  I 
visualized/thought  about  “pushing”  the  ring  to  make  it  grow  bigger. 

#2;  Yes,  evoking  &  suppressing  are  very  different. 

Evoking I  think  about  the  color  red  &  try  not  to  think  of  anything  else. 
Suppressing"^  No  actual  thought,  per  se--  just  a  mental  sensation  of 
“blocking”  activity  in  left  hemisphere.  (I  don’t  feel  anything--  I  just 
describe  it  to  myself  in  this  way,  to  remind  myself  what  to  do.) 

#3;  I  did  the  above  consistently. 


3.  The  following  questions  address  your  assessment  of  various  aspects  of  performance  in 
the  color  matching  task. 

(a)  Do  you  feel  you  were/are  more  or  less  able  to  "evoke"  (produce  red)  or 
"suppress"  (produce  blue)? 

#1:  yes 

#2:  Now  I  feel  I’m  better  able  to  suppress.  But  in  early 
training,  this  was  harder. 

#3:  “Evoke”  was  easier. 

(b)  Indicate  which  process  was  harder  based  on  the  following  criteria;  ** 


Evoking  (red) 

No  different 

Suppressing  (blue) 

To  Leam 

o 

To  Initiate 

o 

To  Sustain 

o 

e 

To  Stop 

o 

0 

**Subject  #3  responded  to  only  one  of  the  four  inquiries 

Comments,  if  any: 

#1;  [no  response] 

#2;  My  answers  here  are  “in  general”—  but  these  trends 

sometimes  vary  in  different  sessions  or  due  to  calibration 
effects. 

#3:  [no  response] 
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4.  Did/do  your  strategy(ies)  or  technique(s)  change  over  the  course  of  a  test  session?  If 
so,  in  what  way?  Do  you  have  any  ideas  about  why  this  happens? 

#1:  For  suppress  trials,  I  concentrated  on  making  the  feedback  ring  a 
dot,  which  involved  staring  at  the  center  of  the  square,  and  thinking 
“small.”  For  evoke  trials,  I  visualized  a  large  black  outer  ring  (and 
thus  my  eyes  were  directed  outside  the  outer  ring).  Also,  I 
visualized/thought  about  “pushing”  the  ring  to  make  it  grow  bigger. 
Once  color  match  [was]  achieved,  I  visualized  “letting  go”  to  change 
the  direction  of  the  feedback  ring. 

#2:  My  strategies  usually  don’t  change  over  a  single  session— 
unless  I’m  having  a  lot  of  trouble  &  feel  the  need  to  try  new 
strategies. 

#3:  My  technique  did  not  really  differ  [in]  the  course  of  the  test 
sessions. 


5.  Did/do  you  find  that  your  technique(s)  or  strategy(ies)  change  over  the  course  of 
training?  Please  offer  comments,  if  any. 

#1;  early  on,  tried  to  repeat  thoughts  used  in  flight  task  to  evoke;  this 
strategy,  however,  did  not  work. 

#2:  Yes.  Now  I  know  how  it  “feels,”  and  I  no  longer  have  to  use 
strategies  like  imagery  or  silent  verbal  command.  (I  do  still  think 
about  the  color  red  to  evoke.) 

#3:  My  technique  did  not  change  during  the  course  of  the  training. 


6.  Please  rate  your  present  overall  ability  to  do  the  color-matching  task. 


0  1 

2 

5  6  7  8  9  10 

NO 

CONTROL 

@  o 

PERFECT 

CONTROL 

Comments,  if  any;  [no  responses] 


7.  Concerning  your  present  overall  ability,  please  indicate  the  answer  that 
corresponds  best  to  your  experience. 


A)  I  always  have  to  concentrate  very  hard. 

0  B)  I  usually  have  to  concentrate  very  hard. 

C)  I  sometimes  have  to  concentrate  very  hard. 
O  ©  D)  I  rarely  have  to  concentrate  very  hard. 

E)  I  never  have  to  concentrate  very  hard. 

Comments,  if  any:  [no  responses] 


8.  Questions  (a)  -  (e)  refer  to  how  challenging  you  found  specific  aspects  of  performance 
on  this  task. 

(a)  Initiating  the  brain  response  to  the  neutral  zone  (to  initiate  task  onset)  was: 


CD 

in 

CO 

CM 

8 

9 

10 

NOT  DIFFICULT 
AT  ALL 

o 

© 

VERY 

DIFFICULT 

© 


X 

CANT  RATE 
since  difficulty 
varied 


Comments:  [no  responses] 


(b)  Increasing  the  brain  response  in  an  "evoke"  trial  was: 


1  2  3  4  5  6 

7 

8  9 

10 

NOT  DIFFICULT 
AT  ALL 

©  ©  O 

VERY 

DIFFICULT 

Comments:  [no  responses] 


X 

CANT  RATE 
since  difficulty 
varied 


(c)  Decreasing  the  brain  response  in  a  "suppress"  trial  was: 


naoEiaa 

7  8 

9  10 

NOT  DIFFICULT 
AT  ALL 

O  ©  © 

VERY 

DIFFICULT 

Comments:  [no  responses] 


X 

CANT  RATE 
since  difficulty 
varied 
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(d)  Changing  the  brain  response  toward  the  neutral  zone  to  close  a  trial  after  an 
"evoke"  (red)  match  was: 


1  2  3  4  5 

6  7  8  9 

10 

NOT  DIFFICULT 
AT  ALL 

®  O 

@  I 

VERY 

DIFFICULT 

Comments:  [no  responses] 


X 

CANT  RATE 
since  difficulty 
varied 


(e)  Changing  the  brain  response  toward  the  neutral  zone  to  close  a  trial  after  a 
"suppress"  (blue)  match  was: 


1  2  3  4  5 

o 

o> 

00 

CO 

NOT  DIFFICULT 
AT  ALL 

O  ® 

e 

VERY 

DIFFICULT 

Comments:  [no  responses] 


X 

CANT  RATE 
since  difficulty 
varied 


(9)  Were  there  any  aspects  of  the  task,  or  the  task  display,  that  you  found  confusing  or 
difficult  to  interpret  or  understand?  If  Yes,  please  describe. 

#1:  No 
#2:  No 
#3:  No 


(10)  Do  you  feel  the  feedback  circle  is: 

@  (a)  too  sensitive  (jumpy,  fast) 

O  ®  (b)  about  right 

(c)  too  sluggish  (dragging,  slow) 


Comments,  if  any:  [no  responses] 
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(11)  Did/do  you  have  any  difficulty  detecting  color  differences  or  color  matches?  If  Yes, 
please  describe. 

#1:  Yes,  there  were  two  shades  of  blue  that  I  was  typically  confused 
as  to  whether  I  should  evoke  or  suppress. 

#2:  No 

#3:  No 


(12)  Did/do  you  direct  your  visual  attention  toward  any  particular  portion  of  the  task 
display  or  screen  at  particular  times  during  the  task?  If  Yes,  please  describe  specifically 
what  you  do,  and  when/why. 

#1;  Yes:  at  center  during  suppress,  outside  outer  ring  during  evoke. 

#2;  Yes:  I  do  tend  to  look  at  different  areas  of  the  box  to  evoke 
(border)  vs.  suppress  (center). 

#3:  Yes:  looked  to  right  to  evoke  and  left  to  suppress. 


(13)  Did/do  you  tense  or  move  your  eyes,  muscles,  or  body  during  the  task?  If  Yes, 
please  describe  specifically  what  you  do,  and  when/why. 

#1:  Yes:  some  eye  movement  involved  in  changing  direction  of 
attention.  At  times  I  found  myself  staring,  prohibiting  eyes  from 
blinking. 

#2:  Yes:  some  eye  movement  --  I  do  tend  to  look  at  different  areas 
of  the  box  to  evoke  (border)  vs.  suppress  (center) 

#3:  Yes:  moved  eyes  to  right  to  evoke  and  left  to  suppress 


(14)  Were/are  there  any  distractions  during  the  course  of  the  experiment  (noise, 
temperature,  etc.)?  Were/are  the  scalp-attached  electrodes  distracting  in  any  way?  If  Yes, 
please  describe. 

#1:  No 
#2:  No 
#3:  No 
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15)  Did/do  you  experience  any  of  the  following  during  the  experiment:  eye  strain, 
headache,  mental  fatigue,  physical  fatigue,  postural  discomfort,  or  other  discomfort?  If 
Yes,  please  describe. 

#1:  No 

#2;  Yes:  postural  discomfort;  some  mental  fatigue  during  final  run; 
eye  strain  during  last  half  of  session. 

#3:  No 


(16)  Please  provide  any  other  comments  you  would  like  to  make  concerning  this  task  or 
experiment,  or  concerning  brain-actuated  control.  Possible  topics  include  training, 
instructions,  task  parameters,  calibration,  recording  procedures,  and  experimental 
procedures. 

#1:  [no  response] 

#2:  Need  to  make  calibration  ongoing,  adjustable  to  change  over  time. 

#3:  [no  response] 
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